This invention relates to computer systems; more particularly, to methods and apparatus for transferring data and check information between computer main memory and data storage devices by means of direct memory access.
Computer direct access data storage devices are typically organized into fixed-size blocks of data storage. For example, a computer disk drive may organize data storage into a number of blocks of data, each block holding 512 bytes of data, and each block having a unique logical block address, with the blocks being sequentially numbered starting with block zero. Other block addressing methods are also in use, including specifying cylinders, tracks, and sectors; but again, each block has a unique address, and the blocks are considered to appear on the device in a well-defined logical order. Other storage media such as tapes may also be organized into fixed-size blocks, depending on the application.
A substantial portion of the work carried out by a computer system is the storage and retrieval of data to and from data storage devices (I/O). In early computer systems this work was carried out by the central processing unit (CPU), requiring the use of a substantial portion of the CPU""s available computing capacity to perform the detailed steps of the I/O. However, in modern systems the mechanics of data transfer to and from storage devices is often carried out by a separate direct memory access (DMA) controller. FIG. 1A shows a simplified block diagram of such a system, shown at 101. System 101 has a CPU 103, main memory 105, DMA controller 107, storage medium 109, and communication bus 111. In systems like 101, to initiate the I/O data transfer, CPU 103 programs DMA controller 107 with the information it requires, and issues a command (via communication bus 111) to begin the transfer. The CPU can then attend to other computations while the DMA controller carries out the detailed steps of the transfer. Thus, DMA controller 107 serves as an intermediary between main memory 105 and storage medium 109. When the I/O operation is complete, CPU 103 is informed of the I/O completion by DMA controller 107, whereupon the CPU may make use of the results of the transfer. In this way, most of the work required to carry out the I/O is offloaded from the CPU, thus freeing the CPU to carry out other work. For example, the CPU may perform processes such as interchange with main memory 105.
To program a DMA device, a CPU typically specifies such information as: a start address in main memory to be used for the data transfer, the length of the data to be transferred, whether data should be transferred from the device to main memory (xe2x80x9creadxe2x80x9d) or from main memory to the device (xe2x80x9cwritexe2x80x9d), and for direct access devices, the address of the block of data on the storage device.
Often it is desired to transfer more than one block of data to or from a storage device in a single I/O operation. In most cases the length of such transfers are constrained to be integral multiples of the storage device""s block size. FIG. 1B depicts a scenario 113, wherein data is transferred from main memory 105 to a storage medium 109, via DMA controller 107. If a CPU specifies to DMA controller 107 a length of more than one block of data, the I/O operation transfers data from sequential addresses in main memory, beginning at the specified start address, to sequential blocks in storage medium 109. The data is stored beginning at a specified block address, or in some devices, at an implicit current location on the storage medium. The DMA controller typically has registers for main memory start address, data block length, and optionally a storage address associated with a storage medium. Thus, in this example, a chunk of data 4096 bytes (8-512 byte blocks, designated D0-D7) in length is transferred to storage medium 109 in one operation (stored in 8-512 byte blocks, designated B0-B7). By transferring multiple blocks of data in a single I/O operation, the CPU is relieved of the effort of initiating the DMA I/O operation for each of blocks D0-D7 individually.
Often it is necessary or desirable to transfer multiple blocks of data between non-contiguous locations in main memory to a contiguous set of blocks on the data storage device. If the DMA controller only accepts a single main memory start address for any given I/O operation, then only sequentially addressable locations in main memory can be transferred in a single I/O operation. In cases where the granularity constraints of the device are met by the organization of the data in main memory, it may be possible to issue separate I/O commands for each contiguous chunk of memory. However, this would incur the CPU cost of initiating the additional I/O commands, and handling their completion.
To alleviate this problem, some DMA controllers implement what has been called a xe2x80x9cscatter/gatherxe2x80x9d I/O operation. In such operations, instead of being programmed with a single main memory start address and length, the DMA controller is given a list of start addresses and lengths. Data is first transferred beginning with the main memory address specified in the first scatter/gather list element. When this element is exhausted; that is, when the specified length of data for the element has been transferred, the next element of the list is used to determine the next set of main memory addresses to be used in the transfer. In this way, a single I/O operation can transfer multiple blocks of data between the data storage device and non-contiguous locations in main memory. However, conventional DMA controllers commonly allow for only a small, limited number of scatter/gather list elements or do not implement scatter/gather capability at all.
In computing environments requiring high reliability data storage and retrieval, it may be necessary or desirable to keep check information for each block of stored data, which can be used to validate that the data has not become corrupted during storage. Although there are many types of check information, one common form is called a checksum. A checksum is the result of a mathematical calculation on a block of data, which produces the same result each time it is calculated. If a checksum is calculated on a block of data that is about to be stored, the checksum is typically stored along with the data. When the data is retrieved, the checksum is recalculated on the retrieved data and compared with the stored checksum. If the newly computed checksum differs from the stored checksum, then the data is known to be corrupt, and remedial measures can be initiated.
In many cases, it is advantageous for data integrity check information computations to be carried out at a very low level in the operating system. For example, these computations may be performed in a device driver, without knowledge of the software entities originally requesting the I/O. The entity requesting the I/O may simply request, for example, that 64 blocks of memory be written to the storage medium starting at a particular block address. The check information is computed and checked by the device driver, and does not appear in the data passed between the requesting entity and the device driver.
Considerable care must be taken in storing data and its associated check information in such a way that they remain consistent on the storage device in the event of a system failure. Significant simplification of this process can be achieved if the check information for a block of data is stored on the device in the same I/O operation that writes that data. One way to do this is to format the storage device so that each block of storage on the device is long enough to hold both the desired data block size, and the desired check information size. For example, if the desired data block size is 512 bytes, and the desired check information size is 8 bytes, then the storage device could be formatted into 520-byte blocks, each of which would hold 512 bytes of data and 8 bytes of check information. In this example the CPU would then issue I/O operations in integral multiples of 520 bytes, with the first 512 bytes being the data for the first block in the I/O, the next 8 bytes being the check information for the first block in the I/O, the next 512 bytes being the data for the second block in the I/O, the next 8 bytes being the check information for the second block in the I/O, and so on.
Currently, implementing check information storage in longer storage block sizes presents a problem. For example, in writing multiple data blocks in one I/O operation, the check information computed by a CPU (for example in a device driver) must be interleaved with the data blocks. As well during a read operation, the interleaved check information must be extracted from the interleaved data, so the device driver can deliver to its clients data free of check information. Even when writing only a single block, the check information must also be included with the data during the write. However, the data arriving to the device driver from the requesting entity does not contain this check information, nor is space reserved for it.
To accomplish a write operation, if the DMA controller does not have scatter/gather capability, then the data must be copied from the main memory locations, provided by the requesting entity, into another buffer that is long enough to hold not only the data, but also the check information for that data. For example, if the requesting entity passed a 512-byte buffer, and the check information comprises 8 bytes, the device driver of the DMA controller must: allocate a 520-byte contiguous buffer, copy the data into it, compute the check information, and place it after the data in the 520-byte buffer. The device driver can then program the DMA controller to carry out the 520-byte I/O operation using the local buffer. For another example, if the device driver is passed a 4096-byte page, the data must be copied 512 bytes at a time into a 4160-byte local buffer, with each 512-byte chunk being immediately followed by its corresponding check information computed by the device driver. However, such copy operations place a burden on the CPU that could otherwise be utilized for other computations.
Even if the DMA controller has scatter/gather capability, implementations often limit the length of the scatter/gather list. Attempting to use scatter/gather capability to interleave check information computed by the device driver with data for a given I/O request may exceed the limits of the scatter/gather list, again requiring data to be copied, or requiring I/O requests to be broken down into multiple smaller requests. For example, in order to write 16 4096-byte pages of data with check information to a storage device (formatted with 520-byte blocks, each comprising 512 bytes of data and 8 bytes of check information), would require 256 scatter/gather list elements, rather than the maximum of 16 required to write those same blocks without the check information (8 blocks per pagexc3x9716 pagesxc3x972 elements per block). Thus, standard DMA controllers having scatter/gather capability do not, in many instances, have sufficient storage to implement a scatter-gather list capable of storing information for data and its associated check information once computed by the device driver. Providing more storage capability to the DMA controller may overcome this problem, but at increased cost in computing efficiency.
What is needed therefore are more efficient methods and apparatus for using direct memory access (DMA) to transfer data and associated check information to and from storage media.
The present invention provides efficient architectures and methods for using direct memory access (DMA) to store and retrieve data and associated check information in fixed-size blocks on a data storage device.
In accordance with one aspect of the invention, the DMA controller obtains check information from computer main memory rather than computing the check information internally in the DMA controller. For instance, the check information may be computed by the CPU rather than the DMA controller. In accordance with some embodiments of the invention, the DMA controller implements additional registers to specify, for example, the location of check information in computer main memory. The DMA controller itself carries out interleaving of data with its associated check information as it transfers information between main memory and the storage device. To accomplish this, the DMA controller is given instructions for how the interleaving is to take place. In some embodiments of the present invention, the data block length and check information block length to be interleaved for storage are programmed by the CPU. In other embodiments, one or both of these lengths may be implicit in other programmed parameters and/or the formatting of the storage device.
In accordance with another aspect of the invention, in cases where there is no scatter/gather capability, a DMA controller of the invention accepts two main memory addresses corresponding to the check information and the data to be interleaved. More particularly, in accordance with some embodiments, two different start address registers are implemented to store start addresses corresponding to the check information and the data to be interleaved. In other words, the first start address register specifies the start of the data, and the second start address register specifies the start of the check information for that data. In another embodiment, a DMA controller of the invention provides two additional registers specifying how information taken from the two sets of addresses should be interleaved. One such register specifies the data block size to be interleaved within storage on the storage device, while the other register specifies the check information block size to be interleaved within storage on the storage device. In the latter case, after the DMA controller is programmed and the operation is initiated, the controller steps through the data and the check information using the two address registers, writing the first data block and its corresponding check information, then repeating the operation for the second data block, and so on.
In accordance with yet another aspect of the invention, each I/O operation includes the transfer of a number of data blocks. The CPU computes the check information for each block and places each block""s check information contiguously one after the other in a check information segment of memory (e.g., buffer) prior to programming the DMA controller. By arranging the check information for all data blocks in one contiguous segment of memory, a single address can specify all the check information for all the data blocks in the I/O operation. By designing the DMA controller to interleave the check information obtained from main memory with the data, no additional scatter/gather elements are required, and neither the data nor the check information need to be copied into local buffers of the DMA controller.
In accordance with yet another aspect of the invention, scatter/gather capability is supported. In some embodiments of the present invention this capability is used to access the data only, and a single check information start address is used to access all check information for a given I/O operation. In these cases, the device driver places all check information for the I/O operation contiguously in a single buffer, while the data is accessed using the scatter/gather list. Other embodiments of the invention implement scatter/gather capability for both data and check information.
In accordance with yet another aspect of the invention, during read operations, DMA controllers of the invention use the above mentioned registers to deconvolute stored data and associated check information into its constituent parts. The data and associated check information is thus extracted from the storage device using a set of interleave criteria (for simplicity the same term is used for the criteria used to interleave and to deconvolute data and associated check information). The separated data blocks and associated check information, are written into a memory external to the DMA controller (for example main memory).
These and other features and advantages of the present invention will be described in more detail below with reference to the associated figures.